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Abstract 

Background: Decision Analytic Models (DAMs) are established means of evidence-synthesis to differentiate 
between health interventions. They have mainly been used to inform clinical decisions and health technology 
assessment at the national level, yet could also inform local health service planning. For this, a DAM must take into 
account the needs of the local population, but also the needs of those planning its services. 
Drawing on our experiences from stakeholder consultations, where we presented the potential utility of a DAM for 
planning local health services for sexually transmitted infections (STIs) in the UK, and the evidence it could use to 
inform decisions regarding different combinations of service provision, in terms of their costs, cost-effectiveness, 
and public health outcomes, we discuss the barriers perceived by stakeholders to the use of DAMs to inform 
service planning for local populations, including (1) a tension between individual and population perspectives; 
(2) reductionism; and (3) a lack of transparency regarding models, their assumptions, and the motivations of those 
generating models. 

Discussion: Technological advances, including improvements in computing capability, are facilitating the 
development and use of models such as DAMs for health service planning. However, given the current scepticism 
among many stakeholders, encouraging informed critique and promoting trust in models to aid health service 
planning is vital, for example by making available and explicit the methods and assumptions underlying each 
model, associated limitations, and the process of validation. This can be achieved by consultation and training with 
the intended users, and by allowing access to the workings of the models, and their underlying assumptions 
(e.g. via the internet), to show how they actually work. 

Summary: Constructive discussion and education will help build a consensus on the purposes of STI services, the 
need for service planning to be evidence-based, and the potential for mathematical tools like DAMs to facilitate 
this. 
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Background 

Health services consist of collections of complex clinical 
pathways for many patients, and as such, are challenging 
to plan. Compared to service delivery for individual 
patients, for which evidence-based approaches are well- 
established [1], there is less of a tradition of evidence- 
based service planning. For example, evidence-based 
guidelines exist for treating an individual presenting with 
certain symptoms, but it is unclear how to plan appro- 
priate combinations of services to meet the varying 
needs of local populations. Even individual clinical ser- 
vices, such as hospitals, are often planned on the basis 
of limited evidence and without making assumptions ex- 
plicit [2]. Planning services for infectious diseases, in- 
cluding sexually transmitted infections (STIs), is still 
more complex, since each case may or may not produce 
further cases [3], such that the goals of early detection 
and treatment are not only to improve the health of the 
individual but that of the wider population too [4] . 

Currently, evidence-based STI service planning for lo- 
calities may begin with a sexual health needs assessment 
of the local population, typically involving collation and/ 
or collection of (largely) quantitative data, and expert 
opinion, together with descriptions (or mapping) of 
current provision [5]. However, seldom is this informa- 
tion then used explicitly or directly to inform the vol- 
ume of different types of services that should be 
provided to meet the population's needs, often because 
service planners are unclear as to how this can be done 
[6]. Yet various tools have been used to help policy- 
makers choose between health technologies, and these 
could be developed to assist service planning in this 
health area. Decision Analytic Models (DAMs) are one 
type of tool and are well-established in Cost Effective- 
ness Analysis (CEA) [7] as part of Health Technology 
Assessment [7-10]. 

A DAM is a mathematical method of synthesising evi- 
dence on the outcomes and costs of alternative, mutually 
exclusive, healthcare interventions [10]. As an example, 
a simple generic DAM could be one that combined evi- 
dence on resource use (e.g. staff time) with price infor- 
mation (e.g. salary) to show the costs of each of the 
alternatives available to a decision-maker. A DAM ap- 
plied to service delivery would need a more complex 
structure as it needs to take account of the costs of 
current and/or planned service provision, the features of 
these service provision alternatives (e.g. the numbers 
and characteristics of patients attending the different 
types of services in the locality), along with indicators of 
population need (e.g. demographic characteristics, dis- 
ease prevalence) for public health outcomes, such as the 
incidence of disease. While such models are computa- 
tionally demanding, increases in computing power are 
facilitating the consideration of such complexities. 



The synthesis of different types of data is an attractive 
feature of a DAM, although it is important to recognise 
that DAMs do not act as a substitute for the best avail- 
able evidence, and as a model, a DAM is only as good as 
the data it synthesises. Other attractive features include 
that a DAM's inputs, assumptions, and model structure 
can all be made explicit [11], and the results of sensitiv- 
ity analyses (the effect on outcomes of changes in input 
parameters [12,13]), can be published, revealing how the 
model works and how robust it is. 

The MSTIC study, an abbreviation for 'Maximising 
STI Control in local populations', funded by the UK 
Medical Research Council (grant number G0601685) 
sought to develop a DAM to be used by planners of 
local STI services. Specifically, we aimed to develop a 
tool to compare the impacts of different combinations of 
specialist, genitourinary medicine (GUM) services and 
primary-care based services, on STI incidence and the 
associated costs and cost-effectiveness. The DAM was to 
be based on a model of STI transmission that took into 
account demographic characteristics, for example, the 
age-sex structure of a local population [13]. 

We assumed that users of the DAM would not have 
specialist knowledge of health economics or epidemio- 
logical modelling so we planned to present the results of 
the DAM as an interactive web-based tool. While several 
health-related web-tools are publicly available and in use 
(e.g. QRISK B> 2 [14]), they are novel in service planning 
[15]. We recognised that our web-tool should not re- 
quire a high level of computing expertise or a steep 
learning-curve for its use, and so we planned that users 
would simply input data for their locality into the tool, 
with online help resources available. The web-tool would 
process these data using the results of the underlying 
model to generate outputs. By varying the data entered 
for different parameters, users could view the epidemio- 
logical and economic implications of changing, for ex- 
ample, the overall capacity or relative capacities of 
specialist and primary-care based services. We planned 
to include a technical appendix in the web-tool's user 
guide that stated all the assumptions and parameter esti- 
mates used in the underlying DAM to promote transpar- 
ency and, we hoped, trust in the web-tool. 

To explore the extent to which our DAM, presented 
as a web-tool, would be considered by stakeholders as a 
useful and acceptable adjunct to service planning, we 
sought the views of service providers, public health spe- 
cialists and commissioners working in sexual health. We 
presented our research plans for developing a DAM and 
web-based tool to assist the planning of STI services for 
local populations, as described above, to a workshop on 
the sources and uses of sexual health data, held in 
March 2009, comprising approximately 80 clinicians, 
public health specialists and commissioners, all with a 
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responsibility for, or surveillance interests relating to, 
STI services in one region of England. Data collection 
was by means of notes taken during a plenary discus- 
sion, and we solicited stakeholders' views by inviting the 
participants to discuss the tool with us after the discus- 
sion (face-to-face or by email after the event), and 
provided sticky notes to be stuck onto a flip-chart - 
conduits for public and more private feedback. In this 
paper, we describe the themes that emerged rather than 
present direct quotations, as we did not request partici- 
pants' permission to quote them, nor did we record the 
workshop. 

Here we present the major themes expressed that were 
of interest and concern for the stakeholders. We then 
discuss implications for the development, dissemination 
and potential use of DAMs in the planning of local 
health services. 

Discussion 

First we present reactions emerging from our initial 
presentation of our research plans. We then discuss pos- 
sible reasons for these, and considerations for others 
attempting similar work in this area. 

Reactions and responses to the concept of a DAM for STI 
service planning 

Views expressed in the plenary were generally sceptical 
about the relevance and utility of a DAM for STI service- 
planning. However, following this discussion several indi- 
viduals privately expressed support for our proposed 
DAM and web-tool and offered constructive feedback. 
Three key themes emerged from the workshop: 

1) Tension between the individual and population 
perspectives 

A number of stakeholders expressed a focus on the 
individual patient's health benefit, and more 
generally the patient's healthcare experience, almost 
to the exclusion of wider public health benefit that 
services might provide, i.e. their impact upon rates 
of infection. As such, some interpreted the MSTIC 
study's objective as directly opposed to providing 
'holistic' patient-centred care, and criticised the 
DAM's lack of assessment of more qualitative 
aspects of healthcare provision because of its focus 
solely on quantifiable inputs and outputs. Given this 
perceived conflict between the individual and 
population perspectives, it was unsurprising that 
some stakeholders were wary of cost-effectiveness 
evidence as well as health economics more generally. 

2) Reductionism 

The proposed DAM was criticised for being 
'reductionist! and some stakeholders argued that it 
overlooked the complexity of service provision and 



the 'real world'. For example, some felt that the 
DAM's consideration of the different types of STI 
service that may be available in a locality was over- 
simplified and that it did not capture the variability 
in service provision, for instance between different 
GUM clinics and between different general 
practices. 

The 'reductionist' criticism also applied to the 
DAM's inputs and outputs. In our presentation, we 
explained that in a DAM's development, inputs can 
be varied within realistic ranges in order to 
determine which are most influential on the outputs 
(sensitivity analyses) [11,12] and conversely which 
data inputs are relatively unimportant and so need 
not be collected. We thought that the volume of 
information service planners can reasonably be 
expected to gather, as well as the ease with which 
they can do so, would be important considerations. 
Consequently, we perceived the reduced burden of 
data collation associated with using a DAM as an 
attractive feature. However, a number of the 
audience interpreted the concept of making 
decisions based on a reduced set of variables as less 
evidence-based. Indeed, some participants later in 
the event called for more data collection to inform 
improved service planning. 
3) Lack of transparency 

Formulation of a DAM involves building a structure 
that synthesises key inputs, which, as we explained 
to the audience, we planned to present as an 
interactive web-tool to make the DAM easily 
accessible. We perceived this as an additional 
strength of our model. However, our audience was 
concerned that the DAM would be a 'black box' and 
called for the model's workings not to be concealed. 
Our perceived motives for developing a DAM to 
assist in STI service planning were also a source of 
criticism. For example, our plan to incorporate cost- 
effectiveness evidence into our model was viewed by 
some as an underhand threat of service cuts, or as 
providing evidence that could be exploited either for 
that purpose or to redistribute resources between 
services. However, we observed a public/private split 
in expressed attitudes to the proposed DAM, as 
several participants approached us privately later in 
the event expressing support for the use of health 
economic methods in service planning and engaged 
in discussion about the assumptions a DAM could 
reasonably make. 

Why are quantitative, model-based tools for service 
planning helpful? And why are they not more popular? 

The increasing diversity of health services providing care 
for STIs in the UK [16], together with the dynamic 
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nature of STI transmission [3], make it increasingly im- 
portant to assess the impact of different combinations of 
service provision with varying clinical outcomes on STI 
control in local populations. While tools based on such 
quantitative evidence can be used to make explicit com- 
parisons between service configurations for a locality, 
our consultation found that tools like a DAM may be 
perceived as threatening and reductionist by clinicians, 
local public health staff and service commissioners. This 
may reflect epistemological differences in the ways in 
which we, as academic researchers, and our stakeholders 
approach STI service planning, including what constitu- 
tes valid evidence, and how this evidence should be 
brought together. An example of this difference was the 
particular tension we observed between the individual 
and public health perspectives, reflecting the differing 
goals of the clinician and public health advocate [17]. 
Another example is a misunderstanding of the use of 
health economics, as others have discussed [18], which 
may in part reflect cynicism in the way the language of 
economics is perceived to be used by public figures 
attempting to justify cost-cutting, and not simply a lack 
of understanding of the discipline. 

Lack of understanding of models may contribute to 
health professionals' reluctance to follow their recom- 
mendations [19], and this can apply to DAMs as well as 
other mathematical models. Furthermore, where a tool 
originates outside a user's discipline, and he or she may 
not know (or trust) the interests of those who created 
and funded the model, this can also contribute to mis- 
trust in the results [20]. Yearley [20] reported that public 
understanding of computerised scientific models 
depends on the trustworthiness of the institutions devel- 
oping models, the public's own knowledge (and lack of 
recognition by scientists of this), and the evaluation of 
assumptions within models (which may indeed not be 
appropriate), which can override the credibility and ap- 
peal that models might otherwise have. As publicly- 
funded scientists working on the MSTIC study, we must 
clearly emphasise to potential users of our web-tool that 
we have no stake in the cutting or funding of particular 
types of service - but it remains unclear how much re- 
assurance our separation from decision-making will give. 

As noted above, assumptions made and data used by a 
DAM can limit its validity - yet these potential limita- 
tions can apply to other methods for planning services 
too. Where DAMs (and other mathematical models) 
contain parameters which are poorly understood and 
therefore particularly uncertain, for example, transmis- 
sion probabilities, then different models of the same out- 
come can result in very different predictions, as was the 
case with predicting the population impact of chlamydia 
screening [9,21-23]. Together with the media's coverage 
of models that have been less successful in their 



predictions (e.g. climate change modelling, swine flu 
modelling) [24,25], this is likely to exacerbate health pro- 
fessionalsj and more generally, the public's mistrust in 
the validity and utility of modelling. Further robust val- 
idation of tools such as DAMs for service planning, in 
any case a valuable exercise, may enable us to advocate 
more confidently their use in this area, and perhaps allay 
some of the scepticism of clinicians and commissioners. 
Nevertheless, among those whose reservations about 
DAMs for service planning are more fundamental (mis- 
trust in health economics and/or a quantitative approach 
to planning, for example), it is unclear whether further 
validation will be convincing. 

We note that DAMs do not constitute 'research evi- 
dence' in the empiricist tradition (such as the results of 
randomised controlled trials or service evaluations). Nei- 
ther do they 'fit' within a conventional step in the commis- 
sioning cycle [26]. Unsurprisingly, the UK's National 
Institute for Health and Clinical Excellence (NICE) has 
produced guidance informing the commissioning of ser- 
vices for various types of cancer [27], but not for STIs or 
indeed any infectious diseases. The potential role of 
DAMs is, as we have seen, untested and their use under- 
developed at the present time and, as discussed earlier, less 
straightforward for infectious diseases. The relative nov- 
elty of using models for STI service planning may there- 
fore have made it difficult for stakeholders to engage with, 
and discuss, our proposed DAM and web-tool. 

There is however a clear need to improve evidence- 
based service planning as current efforts involve colla- 
tion (and often collection) of a considerable amount of 
data, combined in ways which may be neither transpar- 
ent nor, in effect, any less reductionist. This is not only 
time-consuming but there is also evidence of a large du- 
plication of work between those planning services for 
different localities in the same health areas [28]. It was 
therefore interesting to see that our proposal to use a 
DAM, which would reduce the amount of data required 
for evidence-based planning by identifying only key 
parameters, was unfavourably received in our consult- 
ation. On the contrary, there were calls for more data 
collection rather than evidence-based methods of using 
existing data. 

The difference we observed between publicly and pri- 
vately expressed views on the DAM is telling, and may 
reflect tensions between roles, particularly between clini- 
cians and those in managerial/ planning roles [29]. While 
ultimately it is commissioners of local services, within 
budgetary and other constraints, who must decide which 
combination of services to fund [28], there is a need to 
acknowledge the objectives, expertise and role that clini- 
cians and public health leads play. This though can cre- 
ate conflict when commissioners consult local experts in 
sexual health while also considering cutting or 
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expanding these individuals' services. Explicit compari- 
sons of cost-effectiveness when assessing the public 
health impact of services, for example, through using a 
DAM, may raise awkward questions to the different par- 
ties in these collaborations, particularly around funding 
allocation. Central to evidence-based planning of service 
delivery is therefore the creation and nurturing of mean- 
ingful dialogue among clinicians and managers [30], to- 
gether with commissioners. This will be increasingly 
important given changes to the commissioning of health 
services specified in the Liberating the NHS White Paper 
[31], as well as in the context of public spending cuts. 

Summary 

It is apparent that there is a need for more and better 
dialogue between health professionals on the one hand, 
and epidemiologists and health economists on the other, 
to improve understanding of the benefits and limitations 
of mathematical decision-making tools such as DAMs. 
For example, while our approach was perceived by some 
stakeholders as reductionist, the fact needs to be con- 
veyed that all service planning methods make simplifica- 
tions and assumptions, and are reliant on the accuracy 
of the available data. Simplifications are not always justi- 
fied or even made explicit, so that detailed critique and 
discussion is lacking. Ongoing consultation is therefore 
essential for the development and implementation of 
novel technology and methods. With an increasing em- 
phasis in publicly funded research on how research feeds 
into public policy and practice [32], this is an important 
skill for public health researchers to develop. 

Of equal importance is the need for continued learning 
among clinicians and decision-makers regarding the na- 
ture and value of evidence and methods from different dis- 
ciplines. Researchers can clearly play a part in this as well, 
for example, by using innovative methods for explaining 
health economic methods and cost-effectiveness (e.g. 
Democs [33]), and not only by demonstrating the validity 
of DAMs for service planning but by communicating how 
validation has been undertaken. We also believe that 
allowing users easier access, such as through our web-tool, 
is vital if DAMs are to be transparent. Thus, constructive 
discussion and education will help build a consensus on 
the purposes of STI services, the need for service planning 
to be evidence-based, and the potential for mathematical 
tools like DAMs to facilitate this. 
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